SYSTEM AND METHOD FOR FINGERPRINT IMAGE ENHANCEMENT 
USING PARTITIONED LEAST-SQUARED FILTERS 



FIELD OF THE INVENTION 



The present invention relates to the area of fingerprint image processing and, in particular, to 
5 enhancement of the input fingerprint image for aiding in visual examination of various 
fingerprint features and computer processing of the input fingerprint for verification and 
identification of the subject. 



BACKGROUND OF THE INVENTION 



Reliable personal identification is critical in many applications in order to prevent identity fraud. 

10 The losses due to identity firaud can be substantial in terms of money and confidentiality. 
Traditionally, Automatic Authentication Systems have been employed in applications such as 
criminal identification, access control systems, and large-scale social service or national identity 
registry applications. With the emergence of new applications in E-Commerce there is a renewed 
interest in fast and accurate personal identification. For example, in applications such as 

15 web-based retailing, autonomous vending, and automated banking, authentication of consumer 

identity is critical to prevent fraud, hi many of the new applications, the amount of time taken for 

identification is an important issue along with accuracy since the applications are real-time. 

While many authentication schemes have been proposed and implemented, biometrics is 

emerging as the most foolproof method of automated personal identification. The relative 
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permanence of physiological/behavioral characteristics, the fact that biometrics cannot be lost or 
misplaced, and uniqueness over a large population have been cited as the distinct advantages 
of biometrics over other authentication mechanisms. Traditionally, fingerprints have been the 
most widely used and trusted biometrics. The ease of acquisition of fingerprints, the availability 
5 of inexpensive fingerprint sensors and a long history of usage in personal identification make 
fingerprints the most acceptable form of authentication. 

A fingerprint is characterized by smoothly flowing ridges and valleys. The patterns formed by the 
alternating ridges and valleys have been verified to be unique to each person over a large 
B population and have been used for personal verification over the past few centuries, primarily 
T 0 by the forensic community. For automated fingerprint verification a compact representation of 
the rich topology of valleys and ridges is desirable. Most automated fingerprint verification 
systems extract features from the fingerprint images and use the feature sets for verification. The 
3 naturally occurring ridge anomalies called the minutiae capture the essential feamres needed for 
Z automated verification. There are two basic types of ridge anomalies - ridge termination and 
'^15 ridge bifurcation. The number of minutiae in a fingerprint varies firom print to print and typically 
a feature extractor reports 30-60 minutiae per print. 



A generic automatic fingerprint identification system (AFIS), shown in Fig. 1 consists of four 
modules (1) image acquisition 100, (2) preprocessing (enhancement) 110, (3) feature extraction 
120, and (4) verification or identification via feature matching 140. Typically a fingerprint is 
20 captured either by scanning an inked impression of a finger or using a live-scan fingerprint 

YOR-2000-0609 2 



scanner. Preprocessing is often necessary to improve the quality of the fingerprint so that 
fingerprint features are properly extracted from the image. Finally, authentication is performed 
by matching the acquired features with those of a database fingerprint. 



An ideal sensed or scanned fingerprint image is characterized by smoothly flowing patterns of 
5 distinct ridges and valleys. There are two prevalent and well accepted classes of fingerprint 
features arising fi'om local ridge discontinuities (also known as minutiae) : (1) ridge endings and 
(2) ridge bifiircations. Feature matching is accomplished by comparing the global and/or local 
distributions of the minutiae properties. Often, the imaging limitations, acquisition condition, 
J age, maintenance of the original mipression, as well as skin characteristics cause the acquired 
10 image to be far fi-om ideal fingerprints. It is, therefore, desirable to enhance the sensed image, 
^ and ensure proper performance of the successive feature extraction and matching modules. 

J 

\ Traditionally, the forensic applications have been the biggest end-users of fingerprint 
J enhancement algorithms, since the important ridge details are frequently obliterated in the latent 
fingerprints lifted fi-om scene of crime. Being able to enhance these details and/or being able to 
15 detect the minutiae can sometimes mean a difference between catching a criminal or not. Also, 
with the increasing emphasis on the identity fi^aud in our highly automated and 
Internet-dependent worid, the civil applications (e.g., access control, transaction authorization, 
social security management, etc.) will need fingerprint enhancement algorithms. In order that 
positive personal identification is achieved using biometrics, it is imperative that every subject be 
20 reliably identified/authenticated using the given choice of biometrics. Some kind of exception 
processing based (e.g., possession or knowledge) authentication for poor fingerprints poses threat 
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to the integrity of the system and defeats the purpose using biometrics based authentication. 
Consequently, an APIS should be able to make the best out of the available data and be able to 
match every (e.g., you can not afford to annoy a bank customer by asking multiple presentations) 
biometrics measurement as it is being offered. It is widely known that at least 2-5% of target 
5 population have poor-quality (in oriented sinusoid sense) fingerprints: fingerprints that cannot be 
reliably processed using automatic image processing methods. We suspect this fraction is even 
higher in reality when the target population consists of (i) older people; (ii) people who suffer 
routine finger injuries in their occupation; (iii) people living in dry weather or having skin 
problems, and (iv) people who can have poor fingerprints due to their genetic and racial 
lO attributes. With the increasing demand for cheaper and compact fingerprint scanners, the 
fingerprint verification software cannot afford the luxury of assuming good quality fingerprints 
obtained fi-om the optical scanner. The cheaper semiconductor scanners not only offer smaller 
scan area but also significantiy poorer quality fmgerprints. 



PROBLEMS WITH THE PRIOR ART 



15 An image enhancement problem is, in general, hard to solve. There are two main criticisms 
regarding the performance of existing fingerprint enhancement techniques. First, blocky artifacts 
are generated as the operations are carried out in small windows and then regrouped. Second, 
pixel-by-pixel enhancement takes a disproportionately long time compared to ridge extraction, 
minutiae detection, and featore matching (sophisticated feature estimation may be used to 

20 partially improve the performance but only at the expense of speed). Many existing techniques 
assume that the orientation of the ridges and the fi-equency of ridges sufficiently characterizes the 
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local fingerprint intensity variations. They model local fingerprint intensities as oriented 
sinusoids. One of the main purposes of enhancement is to facilitate the extraction of minutiae - 
but the very assumption of oriented sinusoid for modeHng local fingerprint intensities is violated 
at these minutiae! The same is also true for cutmarks. Essentially the sensitivity parameters 

5 (passband) of enhancement filters provide some nominal performance at these points, while 
introducing blur in other good regions (where models hold true). We believe that a detailed 
description of spatial characteristics of fingerprints (even more true for general images) is 
inherently difficult in the frequency domain. As a consequence, designing good adaptive filters 
in frequency domain is also difficult. Furthermore, there is no simple way to systematically 

0 "tune" the existing enhancement filters for handling various types of sensors and acquisition 
conditions (e.g., at higher noise levels, flatter Gaussians can be used in Gabor filters to reduce the 
filter sensitivity). From the vantage of computation, the enhancement speed is dependent on the 
complexity of feature measurement, and also on the size of filters. Frequency-domain 
enhancement filtering is particularly compute intensive without special-purpose hardware (not 

1 5 an attractive option for desktops and standard workstations). 

The following reference is incorporated by reference in its entirety. 

Y-C Shin, R. Sridhar, S. N. Srihari and V. Demjanenko, 
"Local adaptive contrast enhancement", 
US Patent 5,524,070, June 1996. 



YOR-2000-0609 



5 



In US patent 5,524,0170 to Shin et al., a method to enhance the contrast of a grey-scale image 
locally and adaptively is described. The method has been used in the context of optical character 
recognition. A threshold is used to binarize the grey scale image in order to separate the 
background from the foreground. For each pixel in the image, a local window centered at that 
5 pixel is considered and the mean and the standard deviation of the image intensity in that window 
are determined. If the standard deviation is high, then the pixel is left as it is. When the standard 
deviation is low, a piecewise linear function determined empirically is used to scale the pixel's 
intensity and add an offset to it. The intention is to enhance the contrast of the pixel. The scheme 
is too naive for effective fingerprint enhancement. Contrast stretching alone does not solve the 
i 0 problem of effective fingerprint enhancement. 

4 K-R Peters, "Digital Pixel-Accurate Intensity Processing Method for Image Information 

Enhancement", 
J US Patent 5,592,571, Jan 1997. 

In US patent 5,592,57 1 to Peters incorporated herein, a digital pixel-accurate intensity processing 
15 method for image information enhancement is described. The method separates image 
information into four categories: "large image features", "small special detail", "intensity 
detail", and "image noise". Smoothing eliminates intensity-defmed image components. Detail 
enhancement recovers intensity-defined image components eliminated by smoothing. Piecewise 
linear contrast stretching is used to compensate the reduction in image contrast. In case of 
20 fingerprint images, small details are very important and these details can get damaged during 
smoothing. 

YOR-2000-0609 6 



W. A. Fuss and R. Eschbach, "Image-dependent automatic area of interest enhancement", 
US Patent 5,581,370, Dec. 1996. 



In US patent 5,581,370 to Fuss and Eschbach incorporated herein, a method for 
image-independent automatic area of interest enhancement is described. A histogram is derived 
5 from a selected subset of local histograms representing regions of the image. A filter is applied 
on the histogram to weaken the strong peaks and valleys of the signal describing the histogram. 
The filtered histogram is used for adaptively stretching the contrast of the image. Contrast 
stretching alone does not solve the problem of effective fingerprint enhancement, because it does 
not enhance the image. 

40 

A. Kundu and V. Ramamoorthy, "Image Enhancement System", 
I U.S. Patent No. 5,218,649, June 1993. 

In U.S. Patent No. 5,218,649 by Kundu et al. incorporated herem a system for enhancing the 
quality of an image that suffers from staircase effects, contouring effects, and granular noise is 
15 disclosed, hi this system, the edge and nonedge regions are identified, and then those regions are 
filtered in order to reduce the specific type of quality degradation that generally occurs in the 
specific type of region. The method cannot be extended to fmgerprints since a wider variety of 
local discontinuities occur. 
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Y-T Kim, "Image Enhancement Method and Circuit Using Quantized Mean-Matching 
Histogram Equalization", 

U.S. Patent No. 5,937,090, Aug. 1999. 

An image enhancement method using histogram equaUzation of an image signal is described in 
5 U.S. Patent No. 5,937,090 to Kim for displaying the said signal. The disclosed method utilizes 
cumulative density function of a quantized version of the said image signal. Although histogram 
equalization improves the visual quality of fingerprint images, it does not deal with structural 
damages like cuts, pores, etc. Again, this method does not enhance the image. 



C. Chatterjee and V. P. Roychowdhury, "Methods and Apparatus for Enhancing Gray 
^0 Scale Images", 

U.S. Patent No. 5,604,545, Feb. 18, 1997. 

fJI 

S Chatterjee et al. disclosed in U.S. Patent No. 5,604,545 a method for image enhancement where 
the image signal is divided into a set of regions, and a separate linear function is used for each of 
the regions to produce the enhanced image. However, this method is not detail specific enough to 
1 5 handle fingerprint images. 



T. Kamei, "Apparatus For Extracting Skin Pattern Features and A Skin Pattern Image 
Processor Using Subregion Filtering", 
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U.S. Patent No. 6,018,586, Jan. 2000. 

In U.S. Patent No. 6,018,586 to T. Kamei a method for extracting features such as ridge 
directions, ridge line pitches or their combination from skin images is described. A method for 
enhancing the ridges is also described. A plurality of two-dimensional filters is designed 
5 according to kinds of features to be extracted. The skin image is filtered with each of the plurality 
of filters. The contrast/image intensity of each subregion of the skin image is computed and these 
are compared with each other. The filter which gives the highest image intensity is chosen as the 
filter for that subregion and the feature corresponding to the filter is chosen as the feature for that 

f- subregion. There is no learning involved to learn the filters based on the data so fingerprint 

it! 0 image subtleties are not accounted for. 

1^ R. Eschbach and W. A. Fuss, "Image-Dependent Sharpness Enhancement" 
S U.S. Patent No. 5,363,209, Nov. 1994. 

In U.S. Patent No. 5,363,209, Eschbach and Fuss describe a method of sharpness enhancement of 
a color image signal. The said method increases maximum local contrast to a predetermined 
15 value and all other contrast to a proportional level. Contrast stretching alone does not solve the 
problem of effective fingerprint enhancement. 

R. D. Capello and G. R. Mabry, "Method and Apparatus for Contextual Data 
Enhancement", 
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U.S. Patent No. 5,187,747, Feb. 1993. 

In U.S. Patent No, 5,187,747 to Capello and Marby, a method for fingerprint image enhancement 
where enhancement is performed based on intensity distribution along the direction of the 
fingerprint ridge with respect to the local mean intensity, and distribution of orientation vectors 
5 along the direction of the fingerprint ridge is described. A problem here is that the filters are not 
tuned to minutiae features. 

These limited models and methods, however, cannot be very effective in enhancing poor-quality 
5 fingerprints resulting firom pressure variation, smudginess, dryness and so on. Also, fixed sets of 
m filter coefficients cannot be effective for wide kinds of fingerprints which are acquired using 
^ 0 different sensor characteristics, population etc. 

hi O. Oyamada, "Method and apparatus for producing an enhanced two grayscale image", 

O US Patent no. 5,832,123, Nov. 1998. 

In US Patent no. 5,832,123 to Oyamada, a method for binary image enhancement so that they 
are less jagged and more smooth than those in the original input image is outlined. This work is 
1 5 related to binary image enhancement and is not applicable to the grayscale fingerprint images. 

Y. S. Ng and M. A. Pickup, "Resolution enhancement system for combined binary and 
grayscale halftone images", 

US Patent no. 5,600,76 1 , Feb. 1 997. 
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In US Patent no. 5,600,761 to Ng and Pickup a method for determining if the data is grayscale 
or binary, and if it is binary, doing binary edge enhancement is described. The enhancement 
algorithm is appUcable to binary image enhancement and is not useful for the grayscale 
fingerprint images. 

5 R. Loce and M. S. Cianciosi and J. D. Kingsley, "Image resolution conversion method 

that employs statistically generated multiple morphological filters", 
US Patent no. 5,579,445, Nov. 1996. 

Loce et al. introduce a method for automating the design of optimal filters for print quality 
enhancement of binary images. The enhancement algorithm is only applicable to binary image 

10 enhancement. 

•-^ 

M. Mancusso, R. Poluzzi and G. Rizzotto, "Fuzzy logic filter for reducing noise and 
sharpening edges of digital image signals" 
US Patent no. 5,757,977, Aug. 1998. 

Mancusso et al. teach a method of detecting and reducing noise using a fuzzy rules, involving 
1 5 enhancing the edges with no specific details for fmgerprint images. 
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K-R Peters, "Digital Pixel-accurate intensity processing method for image information 
enhancement", 

US Patent no. 5,715,334, Feb. 1998. 

Peters uses selectively smoothing an image pixel with the neighboring pixel(s) if the neighboring 
5 pixel(s) fall within a (user specified) range of grayscale values with respect to the given pixel 
grascale value. Again, this method does not take advantage of the underlying structural properties 
of fingerprint images. 

li These limited (general purpose) models and methods, however, cannot be very effective in 
enhancing poor-quality fingerprints resulting from pressure variation, smudginess, dryness and 
II 0 so on. Also, a fixed set of filter coefficients cannot be effective for the wide kinds of fingerprints 
which are acquired using different sensor characteristics, population etc. Further, most of the 
, enhancement prior art is based on sharpening edges which is not always desirable (e.g., a 
1 fingerprint cut is an edge which is an undesirable artifact for automatic fingerprint matching). 

Except for 5,579,445 (Loce et al), none of the enhancement algorithms can be easily trained or 
15 learn automatically to adapt to finger image characteristics of a given fingerprint image 
population. None of the above approaches, except for 6,018,586 and 5,187,747, solve the 
enhancement problem in the context of fingerprints because contrast stretching is just one of the 
many approaches that fingerprint enhancement. That is, contrast stretching is not a fingerprint 
specific approach. 
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The following references are incorporated by reference in their entirety: 



P. Bakker, L.J. van Vliet, and P.W. Verbeek, "Edge Preserving Orientation Adaptive 
Filtering", Proc. CVPR, pp. 535-540, 1999. 

In this paper the authors describe a generalized Kuwahara filter-based orientation-adaptive 
5 method for edge-preserving smoothing of seismic data. This approach does not extend to 
fingerprints where a wider variety of local discontinuities are seen. 

S K. E. Bamer, A. M. Sarhan, and R.C. Hardie, "Partition-based weighted sum filters for image 
T| restoration", IEEE Transactions on Image Processing, vol 8., no. 5, pp. 740-745, 1999. 

The paper describes an image restoration technique involving partitions determined by Vector 
|o Quantization. It requires a priori knowledge of the spectrum of the noise-free image as well as 
1 the spectrum of noise, vdiich is not easily estimated from fingerprint image collections. 

L. Hong, Y. Wan, and A. Jain, "Fingerprint image enhancement: Algorithm and evaluation," 
IEEE Trans, on Pattern Analysis and Machine Intelligence, vol. 20, no. 8, pp. 777-789, 1998. 

15 Here a Gabor filter based enhancement technique is described. Block-wise frequency of the 
ridges is computed. Block-wise ridge orientation is also computed. The filters are tuned to the 
local ridge frequency and ridge orientation. However, frequency and orientation alone do not 
characterize the local neighborhood of fingerprint images, as noted above. 
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T. Kamei, and M. Mizoguchi, "Image filter design for fingerprint enhancement/' Proc, 
ISCV'95, Coral Gables, FL, pp. 109-1 14, 1995. 

Kamei and Mizoguchi defined an energy function for selecting the image features from a set of 
5 bandpass-filtered fingerprint images, according to a smoothness measure. An enhanced image is 
produced using the features that minimize the smoothness functional fi-om the filtered images. 
There is no learning involved from the fingerprint data sets. 

L. O'Gorman, "Matched filter design for fingerprint enhancement," pp. 916-919, 1989. 

O'Gorman proposed a pixel-by-pixel image enhancement technique by application of oriented, 
matched filter masks. He also attempted to quantify and justify the functional relationships 
between image features and filter parameters so that the design process can be easily modified for 
different conditions of noise and scale. However, this method is based on global features that do 
not capture local details and with no learning involved firom the data sets. 

B,G. Sherlock, D.M. Monro, and K. Millard, "Fingerprint enhancement by directional 
1 5 Fourier filtering," Proc. lEE on Vis. Image Signal Process. , vol. 1 4 1 , no. 2, pp. 87-94, 1 994. 

Enhancement of sensed fingerprint images is achieved by global prefiltering in the firequency 
domain using a bank of bandpass filters. The enhanced values at high-curvature regions, i.e., 
near core and delta are obtained by "appropriately weighted" combination of pixel values from 
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two or more prefiltered images. The method generates several enhanced images from which a 
final image is obtained by appropriately combining these images. The method is computationally 
inefficient, and, as noted above, firequency based filters do not represent the minutia detail in 
fingerprints. 

5 All existing fingerprint enhancement techniques are, in principle, based on local estimation of 
ridge orientation, and ridge frequency, followed by convolution with a parametric filter function, 
"tuned" to that orientation and fi*equency. Recent examples of frequency and 
orientation-selective filters include Gabor filter-banks, and directional bandpass filters. Gabor 
JJ^ filters have both firequency-selective and orientation-selective properties and have optimal joint 
[|0 resolution in both spatial and frequency domains. These are parametrically characterized by 
m orientation, firequency, and sensitivity (standard deviations of the Gaussian envelop control 
L sensitivity). Enhancement of sensed fingerprint images is achieved by global prefiltering in the 
fi-equency domain using a bank of bandpass filters in Sheriock et al. The enhanced values at 
n high-curvature regions, i.e., near core and delta are obtained by "^propriately weighted" 
^ 1 5 combination of pixel values from two or more prefiltered images. Kamei and Mizoguchi defined 
an energy fiinction for selecting the image features from a set of bandpass-filtered fingerprint 
images, according to a smoothness measure. An enhanced image is produced using the features 
that minimize the smoothness functional from the filtered images. O'Gorman proposed a 
pixel-by-pixel image enhancement technique by application of oriented, matched filter masks. 
20 He also attempted to quantify and justify the functional relationships between image features and 
filter parameters so that the design process can be easily modified for different conditions of 
noise and scale. 
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OBJECTS OF THE INVENTION 



An object of this invention is an improved system and method for using biometrics. 

Another object of this invention is an improved system and method to improve the visual quaUty 
of the fingerprint images. 

is Another object of this invention is an improved system and method to enhance the visual quality 
I of fingerprints for subsequent visual inspection. 

Another object of this invention is an improved system and method to make the fingerprint 
images more amenable for automatic fingerprint processing. 



SUMMARY OF THE INVENTION 

10 We propose a new approach to fingerprint enhancement using partitioned least squares filters. 
Given a small, but representative, set of fingerprints and the corresponding set of desired binary 
ridge maps (drawn by an expert), a set of adaptive least squares enhancement filters are leamed 
off-line. On-line enhancement is achieved by adaptively convolving the sensed image with 
appropriate filters. Good on-line enhancement performance is achievable when the training set is 
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a good representative of the sensed fingerprint. A single enhancement filter cannot give good 
performance over a large number of sensed fingerprints, because least square estimation, in 
principle, is susceptible to outliers. In order to reduce the error in least squares fit, the sensed 
fingerprint space is partitioned and a least-square filter for each partition is generated. A 
5 hierarchical partitioning scheme is proposed for fingerprint enhancement to simultaneously 
reduce the least squares error and the computational complexity of the enhancement process. 
The partitioning scheme employs image-domain and sensor-domain (e.g., inked, live-scan) 
information, as well as domain knowledge of the subject, e.g., race, sex, age, profession (artisan 
^ vs. not) and so on. Partitioning of orientation and frequency space is performed like in many 
't}0 existing methods. Additional partitions in terms of ridge/valley/transition, amplitude, quality, 
ill and ridge discontinuity are also generated. While sophisticated partitioning may improve the 
Ci enhancement performance, these also increase the computational burden for the on-line 
L enhancement process. On the other hand, a large number of simple partitions can improve the 
H enhancement performance without reducing the speed. Memory considerations for storing filter 
f^5 coefficients are insignificant. 

It is well-known that least squared filters belong to the class of Wiener filter, which is the 
optimal Imear filter. But it requires a priori knowledge of the spectiiam of the noise-fi-ee image 
as well as the spectrum of noise. We circumvent this hmitation by providing the binary ridge 
maps corresponding to a training fingerprint. This may be infeasible for a noisy gray-scale 
20 image, but can be easily performed by a human for fingerprints. 
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BRIEF DESCRIPTION OF THE DRAWINGS 



Figure 1 shows the components of a typical Automatic Fingerprint Identification System (APIS). 

Figure 2 shows the local neighborhood of a fingerprint consisting of alternating ridges and 
valleys. It highlights the fact that ridges and valleys in a local neighborhood are parallel. It also 
shows a window centered at a pixel. 

Figure 3 shows an image neighborhood and enhancement filter definition. It shows an image 
window of size m x n centered at pixel and the corresponding filter window (also of size m x n). 

Figure 4 defines the amplitude of intensities in an image. 

Figure 5 gives the flowchart for enhancing an input fingerprint image using the learned filters as 
a result of the training process. 

Figure 6 gives the flowchart for the filter leaming process. 

Figure 7 gives the flowchart for computing the partition for each pixel of the input image and 
applying the corresponding filter on the pixel. 

Figure 8 gives the flowchart for computing the amplitude partition for a pixel 
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Figure 9 gives the flowchart for computing the frequency partition for a pixel. 

Figure 10 gives the flowchart for computing the quality partition for a pixel 

Figure 1 1 gives the flowchart for computing the ridge/valley/transition partition for a pixel 

Figure 12 gives the flowchart for computing the orientation partition for a pixel 

5 Figure 13 gives the flowchart for computing the ridge/valley discontinuities partition 
computation. 

Figure 14 gives the flowchart for computing the orientation uncertainity partition computation. 
Figure 15 gives the flowchart for input image enhancement. 
Figure 16 gives the flow chart for filter learning process. 
10 Figure 17 gives the flowchart for iterative enhancement of the input fingerprint image. 
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DETAILED DESCRIPTION OF THE INVENTION 



Fingerprint images are extremely rich source of data. The fingerprint characteristics (e.g., ridge 
height, ridge width, quahty) may be significantly different for different target populations. 
Fxirther, the statistics of the fingerprint characteristics of a target population can potentially be 
5 dynamically changing due to any number of extrinsic (e.g., weather) or mtrinsic (e.g., population 
shifts) conditions. Typically, the poor quality fingerprints due to (say) bad weather (e.g., dry) 
conditions may be completely different in appearance than the poor quality fingerprints due to 
(say) bad skin conditions (e.g., excessive sweat). These different poor quality fmgerprints may 
require different and peculiar image processing methods to correctly and effectively enhance 
10 their appearance and to make the fingerprint ridge information more conspicuously visible. 
i Under such conditions, it is desirable that a fingerprint enhancement algorithm can automatically 
adapt to the characteristics of the (poor quality) fmgerprints. Further, different portions of a poor 
quality fingerprint image may have different characteristics (for example, top-left region of a 
fingerprint may be poorly visible due to sweat while the bottom-right portion may be poorly 
15 visible due to a callus). Based on image characteristics of fingerprint portions (sub region), this 
invention automatically and differently processes the different portions of fingerprint image of 
the target population to accurately enhance the detailed information contained in the fmgerprint. 
Further, the correct method of processing for different types of poor quality portions of image is 
automatically learned from example images provided by the system. Because the system learns 
20 the correct processing method from the typical samples/examples, this invention can easily and 
automatically adapt to different poor quality fingerprints from different fingerprint populations. 
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In Fig. 2 a subregion of a gray scale fingerprint image is shown. The brighter region is the valley 
(210) and the darker region is the ridge (220). The transition region(230) is the valley fading in 
to the ridge region. The spacing between the ridge peak and the valley peak is the measure d 
(240) and the orientation (250) of the ridges is the predominant direction of the flow of the 
5 ridges. An oriented filter oflxw (260, 270) is also shown in Fig. 2. These definitions are used 
in the following description. 

The idea underlying the invention is that the different methods of enhancement processing are 
needed for different types of different poor quality images and different types of different poor 
'tf quality image portions. The invention for which the learning part is shown in Fig. 6 describes 
yjo methods for (a) dividing an image into sub regions (620) (also called as blocks, window) (b) 
%0 determining a type (650) (e.g., class, partition) each image/sub region (c) learning the specific 
l. enhancement method (690) corresponding to each type (e.g., class, partition) each image/sub 
5 region from a set of training image/data set (610); (d) for a given poor quality image to be 
C enhanced, applying the specific enhancement method (500) described in Fig. 5, corresponding to 
' 1 5 each type (e.g., class, partition) each image/sub region; 

Learning the specific enhancement method (600) corresponding to each type (e.g., class, 
partition) each image/sub region from a set of training image/data set will be referred to as 
enhancement learning, transformation learning, or leaming of enhancement filter coefficients. 
Applying the specific enhancement method (500) corresponding to each type (e.g., class, 
20 partition) each image/sub region will be referred to as enhancement or transformation or filter 
convolution/application. Thus, first the enhancement methods are learned fi-om a training 
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image/data set and then, these learned enhancement methods can be applied to a given poor 
quality image to be enhanced. Thus, the invention consists of two major steps: (1) enhancement 
learning, transformation learning, or learning of enhancement filter coefficients. (2) enhancement 
or transformation or filter convolution/application. Both of these steps involve (a) dividing an 
5 image into sub regions (520, 620) (b) determining a type (e.g., class, partition) each image/sub 
region (550, 650). Below, enhancement learning, transformation learning, or learning of 
enhancement fiher coefficients step is described first. It is followed by a detailed description of 
the enhancement or transformation or filter convolution/application. 

S More specifically, the learning enhancement step requires a set of example pairs 610 of (poor 
ftp quality, enhanced quality) images. This set is also referred to as trahiing image/data set; the poor 
3 quality images constittUe the input/raw image training set and the corresponding enhanced 
^ quality fingerprint set constitute enhanced image training set. The example pairs can be obtained 
H fix>m an expert who given a set of typical poor quality fingerprint images (input/raw image 
S training set), generates the corresponding enhanced fingerprint images (enhanced image training 
% set) by closely visually examining the poor quality. The training image/data set can also be 
obtained from any other methods including any combination of (a) existing automatic 
enhancement algorithms which have been assessed to perform effective enhancement for given 
set of input/raw image training subset and (b) enhancement by a fingerprint expert. For example, 
a prior art enhancement algorithm A does a good job of enhancing the smudgy poor quality 
20 fingerprints in the input/raw image training set and prior art enhancement algorithm B does a 
good job of enhancing some of the non-smudgy poor quality fingerprints in the input/raw image 
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training set, then algorithms A and B together with a fingerprint expert can employed to obtain 
the corresponding training image/data set. 

Given a (poor quality, enhanced quality) image pair from the training image/data set, first each 
image is divided into sub regions (640). The process of dividing the image into sub regions is 
5 identical for each poor quality training image and enhanced quality image. For example, if the 
poor quality image is divided into 25 square 4x4 non-overlapping subimages, so should be the 
enhanced quality image. Consequently, there is a one-to-one mapping from each poor quality 
image sub region to each enhanced quality image sub region. The division of an image into sub 
Si regions can be accomplished in any number of different methods; each method resulting in 
lb differently shaped individual sub regions, different number of sub regions, and/or different 
3 spatial overlap among the constituent sub regions. For example, a 256x256 pixel size image can 
{ be divided into 4 non-overlapping quadrants of 128x128 pixel size square shaped sub regions, 
B 16 non-overlapping square shaped sub regions, each with a 64x64 size sub regions, or 20 
K overlapping sub regions, 16 of which are of size 64x64 and 4 of which are of size 128x128. 
'"^15 Similarly, an image can be divided into rectangular, hexagonal, or any other shaped sub regions 
which may or may not be spatially overlapping, but should cover the complete unage. 

The division of an image into sub regions may be governed by any number of the local image 
characteristics, global (overall) image characteristics. It may also be governed by the application 
for which the image enhancement is being used, and/or the characteristics of the subjects to 
20 whom the images belong. For example, very smudgy fmgetprint images may be divided into a 
higher number of sub regions than relatively cleaner (good quahty) raw training images. The 
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division of an image into sub regions may be either predetermined either by an expert or by the 
input training set. The exact specification of division of an image into one or more sub regions is 
determined by any one or more of the following: a pixel-wise ampHtude of the sub region, an 
average amplitude of the sub region, an image frequency of the subregion, an orientation of the 
5 sub region, an amplitude transition in the orientation direction, a local contrast, a local phase, a 
scalar quantization of intensity values of the sub region, a vector quantization of intensity values 
of the sub region, a dryness measure of the sub region, a smudge measure of the sub region, a 
quality of the sub region, an average amplitude of the entire image, an average frequency of the 
entire image, an average quality of the entire image, and an average orientation variation of the 
% entire image. In a preferred embodiment, the image divided into a pre-determined number of 
tl non-overlapping square sub regions/blocks of size 16x16; each pixel is represented in exactly 
S one sub region/block. In another preferred embodiment, the unage is divided into a 
^ predetermined number of square sub regions/blocks of size 16x16 and of size 32x32; each pixel 
F^i being represented by two blocks, one of size 1 6x1 6 and the other of size 32x32. 

^5 Once the training set images have been divided into sub regions, each sub region fi-om the 
raw/input training image set is classified into one or more classes 650. The classes can also be 
referred to as partitions. The partitioning or classification of a region into a class can be governed 
by any number of local sub region characteristics, global (overall) image characteristics, global 
(overall) sub region characteristics, the apphcation for which the image enhancement is being 
20 used, and/or the characteristics of the subjects to whom the images belong. Examples of local 
characteristics that can be used for the partitioning include (1) orientation, (2) frequency, (3) 
local contrast, (4) phase, and (5) intensity discontinuity in the direction of orientation vector, (6) 
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scalar quantization of intensity values, (7) vector quantization of intensity values. The 
characteristics 630 used for classification may be one or more of the following: a pixel-wise 
amplitude of the sub region, an average amplitude of the sub region, an image frequency of the 
subregion, an orientation of the sub region, an amplitude transition in the orientation direction, a 
5 local contrast, a local phase, a scalar quantization of intensity values of the sub region, a vector 
quantization of intensity values of the sub region, a dryness measure of the sub region, a smudge 
measure of the sub region, a quality of the sub region, an average amplitude of the entire image, 
an average frequency of the entire image, an average quality of the entire image, and an average 
orientation variation of the entire image. The subject, sensor, working environment 
ft characteristics can also be used for deciding sub region class. As mentioned earlier, 2-5% of the 
ft target population have poor-quahty (in oriented sinusoid sense) fingerprints. We suspect this 
3 fraction is even higher in reality when the target population consists of (i) older people; (ii) 
i people who suffer routine finger injuries in their occupation; (iii) people living in dry weather or 
Fi having skin problems. Also, statistically, fingerprint characteristics are quite different based on 
55 sex or race of a person. Thus, fingerprint space can be partitioned, based on the attributes of a 
^ target population, i.e., age, sex, and race. Also, fingerprint image characteristics vary based on 
the sensor type, i.e., optical vs. livescan, scanned from paper vs. Real-time, as well as acquisition 
environment, e.g., clean vs. usual vs. dirty. Thus partitions can be created for different sensor 
types and acquisition scenario. For each of these partitions, additional partitions are defined 
20 based on the local fingerprint characteristics. 

One particular method of classification of a sub region into one or more classes is by 
classification of the individual characteristic. For instance, say image subregion can be 
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characterized by the sub region image quality and the sub region image contrast. Further, the 
image quality may itself be quantified into three classes (partitions, types or values) of image 
quality (e.g., dry, smudgy, normal) and the image contrast may be quantified into two classes 
(partitions, types, or values) of image contrast (e.g., high contrast and low contrast). 
5 Consequently, the image sub region may be classified into six classes/parititions (e.g., dry-high 
contrast, dry-low-contrast, smudgy-high-contrast, smudgy-low-contrast, normal-low-contrast, 
and normal-high-contrast). In a preferred embodiment, the total number of sub region classes is 
the product of the number of partitions in each mdividual characteristic classes. In another 
preferred embodiment, the total number of sub region classes is related to the number of 
ft partitions in each individual characteristic classes by vector quantization partitioning method 
Jj which described below. A pixel inchided in the sub region may belong to multiple classes if the 
pixel is represented by more than one sub region. 

fi Vector Quantization based Partitioning 

Let A,, A2, An be the attributes that are used for partitioning. For instance (frequency, 
15 orientation, amplitude) could be the three attributes used for partitioning. Suppose we have a 
representative set of fingerprint images chosen from a database of fingerprints. For each pixel or 
image block in this set of images we form an n-tuple (a,, a2, an) where ai is the value of the ith 
attribute for that pixel. Each such n-tuple is a point in an n-dimensional metric space. A 
clustering technique such as the k-means algorithm is used to discover clusters of these points. 
20 Each cluster represents a partition. The cluster center is used for partitioning the pixels of an 
input fingerprint image. Let x be a pixel in the input fingerprint image and let (a,, az, a„) be the 
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n-tuple corresponding to x. Suppose we have identified K clusters. We determine to which 
cluster the pixel represented by (ai, a2, a„) belongs to. 

What particular image/subject/application characteristics (630) to be used for sub region 
classification can be determined by an expert or by an automatic pattern recognition method. For 
5 example, the enhancement learning system may use only image amplitude or it may use image 
quality/amplitude. In the preferred embodiment, the image amplitude, orientation, an amplitude 
transition in the perpendicular orientation direction, image quality, and ridge frequency image 
characteristics are used for sub region classification. 



ffi How many classes/partitions of the each image/subject characteristics be used can be determined 
m by an expert or by an automatic pattern recognition method. In the preferred embodiment, the 
^ number of classes of the image characteristics is empirically determined from a sample training 



set. 



Below, a detailed description of the image characteristics and the individual 
classification/partitioning of these individual characteristics is described. 

1 5 Orientation and Frequency Partitioning 

A preferred way of computing orientation and frequency characteristics is described in the 
following paper: 
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Hong, Y. Wan, and A. Jain, "Fingerprint image enhancement: Algorithm and evaluation," IEEE 
Trans, on Pattern Analysis and Machine Intelligence, vol. 20, no. 8, pp. 777-789, 1998. 



Other prior art methods may also be employed. 



Amplitude Partitioning 

5 Qualitatively, amplitude captures the local ridge-to-valley contrast. AmpUtude at pixel / can be 
estimated by analyzing the projection of the image along a line segment perpendicular to the 
orientation vector and passing through the pixel. Let MIN and MAX be, respectively, the 
* minimum and maximum of pixel intensities in the / x w rectangular wmdow, as shown in the 
ft; Figure 4. The dimension should be set so that at least one ridge/valley pair passes through the 
io rectangle. In one embodiment, amplitude (410) can be simply computed as. 

Amplitude = MAX-MIN 

I Depending on input noise, calculated MAX and MIN, and consequently amplitude may be 

K incorrect. Amplitude estimated is refined by first sorting the intensity vahies in the window, and 

then using say 90% of the data in the middle to make it less dependent on outliers. That is, 

1 5 Amplitude = sortedJist[(int) (0.95 xlxw)]- sortedjistf (int)(0. 05xlx w)J 

Where sortedjistf] is the list of intensity values in the ascending order. In another embodiment, 

a 1-D sinusoidal signal is constructed firom the pixel intensity values in the window and 

amplitude of this sinusoid is calculated. For a wide variety of fingerprints, it is sufficient to 

segment the fingerprint space into 4-6 amplitode partitions (820, 840, 860 880). That is 

20 If(ampitude<tj) (810) 

Amplitude_Partition = / (820) 
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else if (amplitude < tz) (830) 

Amplitude_Partition = 2 (840) 



else if (amplitude < ti) (850) 
5 Amplitude_Partition ~ i (860) 



else (870) 

Amplitude_Partition =iV(890) 

where ti, ti, In are the thresholds for partitioning the amplitude space. These can be 
If) distributed uniformly or chosen such that during training time there are equal number of training 
[15 samples in each partitions. The latter scheme is expected to give better performance in many 
S cases. Amplitude partitioning produces uniform ridge-to-valley contrast in the enhanced image. 

A global threshold can be applied to detect ridges from the enhanced image. Global 
H normalization cannot achieve this performance. In a preferred embodiment amplitude is 
55 partitioned into 3-6 partitions, typically, 4 partitions were used. The amplitude is typically in the 

range [0-255]. So the five partitions are 0-63, 64-127, 128-191, 192-255. 

Frequency Partitioning 

RidgeA^alley frequency is an important structural characteristic of a fingerprint. Locally, the 
ridges and valleys of a fingerprint are parallel. By fi:equency, we mean the number of 
20 ridges/valleys per unit area. Let/denote the fi-equency of an image block. The partitioning of the 
pixels based on fi-equency is achieved as follows: 
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If(f<fi) (910) 

Frequency__Partition = 1 (920) 
elseif(f<fi)(930) 

Frequency__Partition = 2 (940) 

5 

else (970) 

Frequency__Partition = k (990) 

^ HerCj k > 0 is the number of frequency partitions, and 0 <-f] <f2< ... <fkBre positive numbers 
4 representing frequencies. 

10 Quantitatively, frequency captures the notion of ridge width. The width of ridges and valleys 
vary across fingerprints and within the fingerprint. Intuitively, pixels on ridges with different 
widths should be processed differently. Frequency partitioning is intended to do so. In a preferred 
embodiment, fi'equency has been partitioned into 2-6 partitions, typically, 4 partitions are used. 
The fi-equency is typically in the range [0-16]. So the four partitions are 0-4, 5-8, 9-12, and 
15 13-16. 

Quality Partitioning 



Different portions of a typical fingerprint image are of different quality. Quality is a block-level 
characteristic. Let Q(x) denote the quality of the image block to which pixel x belongs. 
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Intuitively, Q(x) falls into three categories: Good, Poor, and Average. A block-level quality 
estimator determines the quality of every image block in the fingerprint. The block-level quality 
takes a value in the region [0,1] with 0 denoting extremely poor quality blocks and 1 denoting 
excellent quaUty blocks. Given a pixel x, it is classified into one of k > 0 quality classes as 
follows: 

if(Q(x) < qi) mo) 

QualityJPartition = 1 (1020) 
else if(Q(x) < q2) (1030) 

Quality_Partition = 2 (1040) 

else (1070) 

Quality_Partition = k. (1090) 
Here, qi, qi, qic are positive constants such that Q<= qi<q2< - <qk = 1-0. 

The intention behind quaUty partitioning is that good quaUty regions should be just smoothed by 
the enhancement filters while poor quality regions should be enhanced. In a preferred 
embodiment, quality has been partitioned into 4-6 partitions, typically, 4 partitions are used. 
Quality is typically in the range [0-1]. So the four partitions were 0-0.25, 0.25-0.50, 0.50-0.75, 
0.75-1.0. 
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Quality can be computed as described in R. BoUe et al. "System and method for determining the 
quaUty of fingerprint images", US Patent US5,963, 656, Oct. 1999. 



RidgeA'alley/Transition Partitioning 

A fingerprint image consists of a collection of smoothly flowing ridges and valleys. The ridges 
5 and valleys alternate and locally they are parallel. A ridge or a valley in a fingerprint image is 
typically 3-7 pixels wide. The intensity of the ridge (or valley) is usually not uniform throughout 
its width. Depending on where actually the pixel lies in the configuration of ridges and valleys, 
5 an appropriate filter should be applied to it. A pixel can lie on a ridge, or on a valley, or on the 
f : transition region. In RidgeA^'alley/Transition partitioning, a pixel is classified as a ridge pixel if it 
Jo lies on a ridge, a valley pixel if it lies on a valley, and a transition pixel if it lies on the 
r ridge/valley transition region. One embodiment of ridge/valley/transition partitioning would 
i classify the pixels in the fingerprint image into three classes, i.e., one class for ridge pixels, one 
!f class for valley pixels, and the third class for transition pixels. However, each such class can be 
^ fiirther partitioned into subclasses if needed. 
15 Let I(x) be the signature value of pixel x and A(x) be the amplitude of the sinusoid passing 
through X. Let MAX and MIN be the maximum and minimum value of the sinusoid passing 
through X. As discussed before, A(x) = MAX- MIN. In the preferred embodiment, we say that 

pixel X is a ridge pixel if 

I(x) < (MIN+0.33xA(x)). 
20 We say that a pixel is a valley pixel if 

I(x) > (MAX- 0.33 xA(x)). 
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If a pixel is neither a ridge pixel nor a valley pixel then we say that it is a transition pixel. 

In a more general setting, we could have k > 3 ridge/valley/transition classes computed as 

follows: 

If(I(x) <= MIN + cixA(x)) (1110) 
5 RVT_Partition = 1 (1120) 

else if(I(x) <= MIN + x A(x)) (1150) 
RVT_Partition = 2 (1160) 



else (1170) 

% RVT_PartiHon= k. (1190) 

hi Here, Ci, C2, Ck are positive constants such that 0<= c, < C2 < ... < <= 1.0. and 
i RVT_Partition is the partition number 1, 2, k. In a preferred embodiment 
l_ RidgeA^alley/Transition has 3-6 partitions, typically 3 partitions. 
ffs Orientation Partitioning 

^15 The ridges and valleys of a fingerprint are locally parallel. They have a dominant orientation m a 
local neighborhood. If the image is divided into blocks of size w x w, where w > 1 (typically, 8 
<^ ^ 32)^ then for each such block the dominant orientation of ridges and valleys can be 
computed. For each pixel x in the image, the orientation 0(x) at that pixel is defined to the 
dominant orientation of the image block to which the pixel belongs. The orientation of a pixel 
20 0(x) takes a value between 0 and 180. This range can be divided into k > 0 intervals with each 
range corresponding to an orientation partition. 
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If (orientation < 01) (1210) 

Orientation _partition = 1 (1220) 

else if (orientation < 02) (1230) 

Orientation _partition = 2 (1240) 

5 

else (1270) 

Orientation _j)artition = k. (1290) 



_ In a preferred embodiment, orientation has been partitioned into 4-12 partitions, typically, 6 
^ partitions are used. The orientation amplitude is typically in the range [0- 1 80] . So the six 
|p partitions are 0-30, 31-60, 61-90, 91-120, 121-150, 151-180. 

Ridge Discontinuity Partitioning 

E Qualitatively, ridge discontinuity indicates the presence of cutmarks, minutiae, and blots in inked 
^ prints. Ridge discontinuity is a pixel-level characteristic. Let x be a pixel. The derivative of the 
image at x in the X and Y directions is determined using the Sobel Operator. The derivative of 
15 the image in a direction is a measure of the rate of change of the intensity of the image pixels 
along that direction. The value of the derivative in a particular direction falls in a range [Dmin, 
Dmax]. This range can be divided into a finite number of subranges and pixels can be partitioned 
based on which subrange they fall in. In the preferred embodiment the derivative Gx along X 
direction and the derivative Gy along Y direction are computed and the partition is determined as 
20 follows: 
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If(G, < G.j) (1310) 

A = ; (1320) 
else if(G, < 6,2) (1330) 

a=2 (1340) 

5 else 

a = 3. 

If(Gy < Gyj) (1315) 

S b = I (1325) 

Z else if(Gy < Gyj) (1335) 

So b = 2 (1345) 

' else 
B b = 3. 

RD_Partition = 3x (a-1) + h. (1397) 

15 Here, Gxi, Gx2, Gyi, and Gy2 are constants lying in the range [Dmin, Dm ax]. 

Instead of the X and Y directions we could use any two dimensional linear space for partitioning. 
For instance, we could use the direction along the ridge and the direction perpendicular to it. In a 
preferred embodiment, ridge discontinuity lies between 4-16, typically 9, and Gx and Gy's are 
uniformly placed between D_min and D_jnax. 
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Orientation Uncertainity based Partitioning 

The ridges/valleys change their direction rapidly at certain portions of the fingerprints and less so 
in other regions. The rapid changes in orientation of the ridges/valleys are prominent at regions 
close to cores and deltas. However, at regions away from cores and deltas, the change in 
5 orientation is gradual. Clearly, these different type of regions have to filtered differently. The 
change in orientation around a pixel can be measured by comparing the orientation around the 
pixel with the orientation of neighboring blocks. Classes are defined on the basis of the amount 
of change in orientation. Let x be a pixel. Let W be a window of size w x w (w > 0) centered at x. 
5 Suppose Wi,..., Wjbe the blocks (each of size w x w) in the neighborhood of W. Let O, Oi,...,Ot 
fi) be the orientation of the ridges in blocks W,Wi,..., Wt respectively. Let U, denote |0 - Oij. O, 
3 measures the deviation of the orientation of block Wi from the orientation of block W. Let Uavg 
= be the average of Ui,..., Ut. 

P: If(Ua.g < U,) (1410) 

^' 0_Uncertainity_Partition = 1 (1420) 

15 elseif (Uavg <U2) (1430) 

0_Uncertainity_Partition = 2 (1440) 

else (1470) 

0_Uncertainity_Partition = k. (1490) 
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Here ui, Uk are non-negative constants chosen appropriately. In the preferred embodiment, the 
values of w, k, Ui, Uk are empirically determined from a set of sample images. In the preferred 
embodiment w=16, k =2, ul=20 degrees, u2=180 degrees. 

So far, the methods of dividing a training (raw, enhanced) image into subregions is described and 
5 the methods of assigning a sub region class/partition to the training raw image sub region based 
on the image/sub region/subject characteristics is described. Application of these methods 
results in zero or more (raw, enhanced) training image subregion associations per sub region 
class/partition. Given a reasonable number of (raw, enhanced) training image subregion 
* associations, the enhancement learning method obtains the transfbnnation parameters for 
jtp transforming the raw sub region of given class/partition into the corresponding enhanced sub 
S region. The method of leammg this transformation is either by a human expert (e.g., visual 
I examination), by automatic signal processing method, or by a combination thereof. In a preferred 
i embodiment, the transformation (filter) are learned completely automatically by a signal 
t processing method called least square estimation filter (Wiener filter), which is described below. 
T'S In a preferred embodiment, orientation uncertainity lies in between 2-6 partitions, typically 4. 
Uncertainity is in the range [0-180], so the four partitions are 0-45, 46-90, 91-135, 136-180. 

Scalar Quantization based Partitioning 

Let X be a pixel. Let Wx be a window of size w x w (w > 0) centered at x. is oriented along 
the dominant ridge direction at x. Let MAX be the maximum intensity value of a pixel in Wx and 
20 MIN the minimum intensity value. We assign a value 1 to each pixel y in Wx if 
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I(y) > (MAX-MIN)/k 



and 0 to each pixel y in Wx if 
l(y) <= (MAX-MIN)/k. 

5 Typically, k is 2. Here, I(y) denotes the intensity value of pixel y. The result of this procedure is 
a binary string of length w x w. This binary string is treated as a number, which gives the 
partition number. In a general setting, only bits in chosen positions in the window could be used 
for forming the binary string and determining the partition number. If L is the length of the 

S binary vector, then there are 2^ classes. In a preferred embodiment, L is in the range 3-6. We 

lljO tried L = 4, i.e., 16 classes. 



Learning Fingerprint Enhancement Filters 

D 



^ Let us define a neighborhood 3 1 0 around a candidate image pixel i from raw training image sub 
region and an enhancement filter 320 to be applied in pixel /, as shown in Fig. 3. In a preferred 

1 5 embodiment, the neighborhood is an « jc m rectangular window. Typically, n is between 1 -5. In a 
preferred embodiment n=3. Typically m is between 3-11. In a preferred embodiment m=7. The 
filter is also defined in the same window. Let us denote h=nm. Now define a vector 



~[x\, 4, .., X},.., 4 ],(640) 



YOR-2000-0609 



38 



Where xj is the image intensity at the >th location within the /-th window. Assume yt is the 
enhanced intensity at pixel /, provided by a fingerprint expert. Now define a matrix 

X=[ Xi, X2, .., X/, .., \M ] 

where, M is the number of candidate pixels fi:om the same sub region partition/class , and T 
5 stands for transpose of a matrix or vector. The idea is to obtain the enhancement filter 

W=[ Wi, W2, .., W/, .., Wk ] 

as an optimal mapping between the sensed raw image space X and the enhanced image space 

Y=[ y\, yi, yj, yu ] - 
3 Thus the filter fiinction w is obtained by minimizing the least squares mapping error 
Ip i^w) =(Xw-Y)^(Xw-Y) 

1 with respect to w. Differentiating F(w) with respect to w and equating the result to zero, the 

Z enhancement filter fiinction is obtained as 

i w=(X^X)''X^Y.(1670) 

t As long as M > it, the existence of (X'rx)-'is guaranteed, practically due to inherent noise 
^15 associated with the imaging process. In a preferred embodiment M is at least 10,000, to ensure 

good performance over different types of fingerprint images. Thus for each sub region class p, 

Wp is learned using the sample sub region association. 

When sufficient samples are not available for a sub region class p fi-om a training set, the training 
set can be augmented to include more images to obtain sufficient samples for the corresponding 
20 class. If eventually, for sub region class p, there are insufficient training samples, transformation 
filter Wp cannot be learned using the sample sub region association and the sub region 
transformation is considered to be invalid. 
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Enhancing Fingerprint images 



The enhancement transformation learning step yields either valid transformation filter Wp (1670) 
for each sub region class p or flags a particular sub region class q to be invalid. 

The method of enhancing a given poor quality image is as follows. The given image is divided 
5 into sub regions (540) using the same process of the image division (640) followed at the time of 

enhancement learning. Each sub region is further classified (550, 1530) using the classification 
% procedure used at the time of enhancement learning. If the transformation filter Wp, 
3 corresponding to a particular sub region, is valid, the filter Wp is applied/convolved with that sub 
I region to obtain the corresponding enhanced sub region. Once Wp is calculated using a 
|0 reasonably large M fi-om a representative set of database fingerprint images, it is convolved with 
1| a sensed fingerprint image, I to generate the enhanced image. Or, mathematically the enhanced 
■3 image is given by 

E = I* Wp(1540) 

1 5 where * denotes convolution operator. 

If the transformation filter Wp corresponding to a particular sub region is invalid, a number of 
alternate methods can be used to enhance such a sub region or the corresponding sub region may 
be left unprocessed. Any one of the following methods of alternate processing can be used to 
process the sub regions belonging to an invalid class: a valid transformation filter associated with 
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a sub region class most similar to the invalid sub region class may be chosen. The metric of 
similarity may be determined using some distance metric (absolute difference, weighted 
Euclidean, Mahalanobis distance metric) defined on the constituent characteristics used for 
determining the sub region class. For example, if the sub region class was determined using the 
5 image contrast class and image amplitude class, a distance metric may be defined on the image 
sub region contrast value and image sub region amplitude value; this distance metric can be used 
to determine the sub region class most similar to the invalid sub region class. Alternatively, the 
image sub regions may computed using a different image division method (e.g., at different 
image resolution) and resultant subregion class may be used for determining the transformation 
■ft filter. Another solution for the invalid sub region class is to use all the transformation filters 
M corresponding neighboring valid subregion classes and either apply them successively or use a 
3 weighted output of the individual convohitions of the transformation filters with the raw input 
s sub region. 

t Similarly, if a pixel belongs to more than one subregions (say, because of the non-overlapping 
I s image division process), there may be more than one candidate transformation filter. Only one of 
the transformation filters may be selected as described above, or all the candidate transformation 
filters may be successively convolved, and/or a weighted summation of the convolved outputs 
fi-om the individual filters may represent the final outcome of the enhancement process. 

Iterative Enhancement 
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The enhancement fihers could also be applied to the fingerprint image iteratively (1710). In one 
embodiment, the original fingerprint image, say lo, is enhanced to get the image Ii; Ii is again 
enhanced to get I2 (1720), and h is further enhanced to get I3. In general, we could apply the 
enhancement filters k > 0 times to get a sequence of enhanced fmgerprint images Ii, h, Ik. 

5 Method to eliminate Orientation Partitioning 

When the number of training images is small, some partitions may have very few samples. In 
such situations, orientation partitioning can be eliminated by a simple scheme. Let x be a pixel 
t and 0(x) be the orientation at pixel x. Let I be a mxn image window aligned along 0(x) and 
centered at x. If w is the filter for pixel x, w is applied on this rotated window I. 

10 Training sample selection procedure 

1 Here is a method to select the training image set automatically when a handful of training images 

3 from a large population of images has to be selected: 

Let Ai, A2, An be the attributes that we want to use for partitioning. For instance (frequency, 
orientation, amplitude) could be the three attributes used for partitioning. For each pixel or image 

15 block in the images we form an n-tuple (a,, az, an) where ai is the value of the i* attribute for 
that pixel. Each such n-tuple is a point in an n-dimensional metric space. A clustering technique 
such as the k-means algorithm is used to discover clusters from these points. Each cluster 
represents a partition. The cluster centers are then used for determining the set of images to be 
used in training. The idea is to select images so that each partition class has an adequate number 

20 of pixels. Therefore, representative images for each partition class are selected. To determine the 
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set of images to be included in the training set for the partition class j (i.e., cluster j determined 
previously), the following process is performed: 

With each image Is in the population, do the following: for each pixel x in I, compute the values 
ai, a2, a„ of the partitioning attributes Ai, A2, An at that pixel and form the n-tuple (ai, 0.2, 
5 a„). Treating (ai, a2, an) as a point in the n-dimensional feature space, we determine the 
cluster in which this point falls and its distance from the cluster center. Suppose K > 0 pixels of 
image li fall in cluster j. Let 6.^ be the average of the distances of these K pixels from the center 
of cluster j. A cost 0 = dji + Sj/K is assigned to image L, where Sj is a non-zero constant. When 
K is small (i.e., only a few points of li fall into cluster j) the term Sj/K dominates and when K is 

% large d,-* dominates C,-*. At the end of this process, we have a set of non-zero values CiJ,... . A 
small fixed sized set of images having the least costs is chosen to be included in the set of 

if i training images. 

5 A preferred embodiment for determining Sj uses Sj to be the average of di',...,di^. To save some 
t computation, the system considers only block centers in the above algorithm, instead of every 
Tii pixel of the images. 



Contrast Stretching of Fingerprint Images 

One way to improve the visual quality of a fingerprint image is to stretch its contrast. However, 
the commonly used contrast stretching techniques do not give satisfactory visual results on 
fingerprint images. The contrast of fingerprint images can be improved significantly by a scheme 
20 that makes use of the local fingerprint characteristics. Such a scheme can also be used as a 
preprocessing step in enhancement. 
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Given below is preferred embodiments of enhancement and filter learning process. Now refer to 
Figure 5. Given an input image to be enhanced and a set of the transformation enhancement 
filters (510), a method for dividing the input image into subregions (520) and a set of 
characteristics to be used for classification (partitioning) (530) of the sub regions is selected. 
5 Subregions (540) are determined based on the subregion division method (520). In step 550, the 
characteristics selected in step 530 are computed for each sub region determined in step 540. 
Based on the selected characteristics of the subregion (550), a transformation filter w is selected 
in step 560. The transformation filter w is applied to the subregion to obtained the enhanced 

S subregions (570). When all enhanced subregions are determined (580), they are merged to 

rllO obtained the entire enhanced image in step 590. 

T Referring to Figure 6, in step 610, the learning enhancement step takes a set of example pairs of 
m (poor quality, enhanced quality) images. This set is also referred to as training image/data set; the 
H' poor quality images constitute tiie input/raw image training set and the corresponding enhanced 
^ quality fingerprint set constitute enhanced image training set. The learning algorithm also selects 
15 a method for dividing the input image into subregions (620) and a set of characteristics to be 
used for classification (partitioning) (630) the sub regions is selected. Subregions (640) are 
determined based on the subregion division method (620). In step 650, the characteristics 
selected in step 630 are computed for each sub region determined in step 640. Based on the 
selected characteristics of the subregion (650), the sub region is held in a different partition/class. 
20 When all subregions are classified (670), the learning process continues the steps 640-670 on a 
different pair of (poor quality, enhanced quality) training image. When all subregions of training 
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image pairs are classified (680), filter parameters for each partition are computed using all the 
subregions belonging to that class (690). 

Refer to Figure 7, it shows one preferred embodiment of enhancement method. Given an input 
image to be enhanced and a set of the transformation enhancement filters, a definition of pixel 
neighborhood (Figure 3), a set of characteristics to be used for classification (partitioning) of 
pixels are selected. For each pixel in the image, its neighborhood pixels are collected (710), and 
based on the neighborhood pixels, the corresponding partition p is determined (720). . Based on 
the partition of the pixel (720), a transformation filter w(p) is selected and applied to the pixel to 
obtain the enhanced value corresponding to that pixel 730. When all enhanced values of all the 
pixels are determined (740), the enhancement method is finished. 

Refer to Figure 15, it shows one preferred embodiment of enhancement method. Given an input 
image to be enhanced and a set of the transformation enhancement filters, a definition of pixel 
neighborhood (Figure 3), a set of characteristics to be used for classification (partitioning) of 
pixels are selected. For each pixel in the image, its neighborhood pixels are collected (1510), 
and characteristics of the neighborhood pixels is determined (1520), a transformation filter w(p) 
is selected (1530) and applied to the pixel to obtain the enhanced pixel value corresponding to 
that pixel 1540. When all enhanced values of all the pixels are determined (1540), the 
enhancement method is finished. 
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Refer to Figure 16, it shows one preferred embodiment of enhancement learning process. For 
each partition p, it starts with initializing a matrix suml(p) and vector sum2(p) to zero (1600). It 
then takes a (poor quality, enhanced quality) of training images (1610). Neighborhood pixels 
(Figure 3) for each pixel (i, j) in the poor quality image is selected and also the desired enhanced 
5 value of the corresponding pixel from the enhanced quality image is selected (1620). hi step 
1630, partition p is determined based on the characteristics of neighborhood pixels of pixel (i, j). 
Then, suml(p) and sum2(p) are updated as shown in step 1640 based on the neighborhood pixel 
values of pixel (i, j) belonging to partition p. When all pixels are visited for a training image pair 
(1650), the learning process continues the steps 1610-1650 on a different pair of (poor quality, 
flo enhanced quaUty) training image. When all pixels of training image pairs are thus visited; suml, 
S sum2 are updated corresponding to the partitions of the pixels (1 660), filter parameters w(p) for 
S each partition p are computed using suml(p) and sum2(p) corresponding to that partition as 
shown in step 1670. 
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